Repository navigation
feat(routing): add opt-in canary and A/B auto-route cohorts - #54
Conversation
Behind CANARY_TRAFFIC_ENABLED, an auto route can carry a reviewed canary policy with fixed baseline and candidate weights. Sessions are assigned to a sticky cohort by an HMAC of the authenticated principal, session, route and salt revision; requests without a session stay on the baseline. Shadow mode records the proposed order without changing dispatch, and live mode puts approved candidates first for the candidate cohort while every existing filter still applies. Responses on opted-in routes carry X-MultiLLM-Canary-Cohort. Canary policies are stored in a new canary_traffic table (migration 0025).
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Deploying with
|
| Status | Name | Latest Commit | Updated (UTC) |
|---|---|---|---|
| ❌ Deployment failed View logs |
multillm-proxy | 6150133 | Oct 09 2026, 02:34 AM |
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configuration
You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Merged the updated |
Summary
Adds opt-in canary and A/B cohorts for auto routes. With
CANARY_TRAFFIC_ENABLEDoff (the default), and for any route without an enabled canary policy, auto-route dispatch, headers and storage are unchanged.canarypolicy together through the existingPUT /admin/auto-routes:enabled,mode(shadoworlive),weights(baseline+candidate= 100),salt_revisionandapproved_candidates. Approved candidates must already be in the route.expected_updated_atlocally,current_revisionwith durable revisions).JWT_SECRET, over["multillm-canary-v1", principal, session, route, salt_revision].X-MultiLLM-Canary-Cohort: <baseline|candidate>; mode=<shadow|live>. Content-free counts by route, cohort and mode are kept per process, up to 800 counters.0025_canary_traffic.sqlstores only the reviewed policy per route and the route's update timestamp. It stores no session, principal or HMAC output. A missing table or an unreadable policy falls back to the baseline with one warning, never a 500.Note: a session ID is a caller-supplied hint, so a caller can change cohorts by changing their session. This is a traffic split, not an access control, and the doc says so.
Details are in
docs/canary-traffic.md.Shared registrations
tests/test_canary_traffic_worker.mjsintest:worker.worker/canary-assignment.mjsin the edge module inventory.CANARY_TRAFFIC_ENABLEDin the Container env keys.canary_trafficinREQUIRED_D1_TABLES.X-MultiLLM-Canary-Cohortin both CORS expose lists.Stacked on #50
Migration numbers must be contiguous in each PR, so this branch is stacked on
feat/context-pages(#50, migration 0024) and targets it. Merge #50 first.Left for the Wave 5 integration
handleCanaryRouteOperationinworker/auto-routes-d1.mjs:handleAutoRoutesRequest, so durable (D1) saves can store policies. Local storage works now.prepareCanaryto native Worker generations innativeGenerationFetch.MetricsService._request_context_metadataandobserveNativeResponse.Tests
tests/test_canary_traffic.pyandtests/test_canary_traffic_worker.mjscover:npm run test:worker: 1,486 passed.npm run test:integration: 7 passed.migrations:check(25 contiguous, additive migrations),test:deploy, free routes, client headers and thewrangler deploy --dry-runbundle all pass.Apply the migration before enabling durable canary routing:
npx wrangler d1 migrations apply multillm-intelligence --remote.CI notes
python/nodejobs: the dependency-audit steps fail on every branch, including main. fix(deps): clear the CI dependency audit advisories #42 fixes that.AdmissionCoordinatorDurable Object migration, whichwrangler versions uploadcannot apply. This is expected untilwrangler deployruns on main.This is part of Wave 5. The README line comes with the Wave 5 integration PR.